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ABSTRACT 

Implicit  in  most  of  what  we  do  in  MIS  is  the  belief  that  information  technology  (IT) 
has  an  impact  on  the  bottom  line  of  the  business.  Surprisingly,  we  rarely  know  if  this 
is  true.  It  is  very  difficult  to  trace  and  measure  the  effects  of  information  technology 
through  a  web  of  intermediate  impacts  upon  enterprise  level  performance.  In  this 
paper,  we  review  research  that  has  been  performed  on  enterprise  level  impacts  of 
information  systems,  with  a  particular  emphasis  on  research  that  has  attempted  to 
measure  those  impacts.  We  begin  with  a  survey  of  articles  published  within  the  last 
ten  years.  This  is  followed  by  a  discussion  of  the  reference  disciplines  that  underlie 
most  of  this  work.  From  this  we  are  able  to  draw  conclusions  about  future  directions 
for  research  in  this  area. 


Introduction 

Implicit  in  most  of  what  we  do  in  MIS  is  the  belief 
/  that  information  technology  (IT)  has  an  impact  on 
the  bottom  line  of  the  business.  Surprisingly,  we 
rarely  know  if  this  is  true.  It  is  very  difficult  to 
trace  and  measure  the  effects  of  information 
technology  through  a  web  of  intermediate  impacts 
upon  enterprise  level  performance. 

When  computer  systems  were  used  largely  for  cost 
displacement,  the  evaluation  of  their  impact  on 
enterprise  performance  could  be  conducted  using 
an  accounting  framework-cost/benefit  analysis. 
The  costs  were  the  costs  of  putting  the  system  in 
place-the  benefits  were  some  offsetting  reduction 
in  headcount  or  other  organizational  costs. 
Currently,  systems  are  often  used  to  enhance 
performance  without  any  necessary  reduction  in 
organizational  costs.  With  some  systems,  the  ben- 
efits are  better  decision  making,  improved  com- 
munications, or  other  semi-tangible  instrumental 
changes.  With  systems  that  try  to  afTect  the  com- 
;  petitive  position  of  the  firm  in  its  marketplace,  the 
;  benefits  are  usually  even  less  tangible. 

The  lack  of  measures  of  enterprise  performance 
impacts  is  a  serious  practical  and  theoretical 
problem.  In  practice,  we  assume  that  our  systems 
will  deliver  bottom  line  value,  but  we  can  neither 
predict  that  value  for  the  investment  decision,  nor 


measure  it  once  the  system  is  in  place.  In  testing 
our  theories  about  effective  information  systems, 
we  are  left  with  surrogate  dependent  variables 
that  may  only  weakly  relate  to  the  ultimate 
measure  of  impact  on  the  firm.  Thus,  we  develop 
theories  about  what  makes  systems  valuable,  but 
we  can  often  only  test  whether  the  systems 
engender  individual  level  satisfaction  or  usage. 

In  this  paper,  we  review  research  that  has  been 
performed  on  enterprise  level  impacts  of  infor- 
mation systems,  with  a  particular  emphasis  on 
research  that  has  attempted  to  measure  those 
impacts.  We  begin  with  a  survey  of  articles  pub- 
lished within  the  last  ten  years.  This  is  followed 
by  a  discussion  of  the  reference  disciplines  that  un- 
derlie most  of  this  work.  From  this  we  are  able  to 
draw  conclusions  about  future  directions  for  re- 
search in  this  area. 


Database  of  Articles  Studied 

A  total  of  11  articles  relating  to  enterprise  level 
performance  was  found  by  searching  10  journals 
from  1975  to  1985.  These  were  felt  to  be  repre- 
sentative in  terms  of  methodologies,  variable  de- 
finitions and  operationalizations,  and  findings  and 
were  used  for  analysis.  Tables  I  and  II  contain  a 
listing  of  these  articles  classified  according  to  the 
type  of  methodology  used.  Empirical  studies  in- 
clude cost/benefit  analyses  and  field  surveys;  the 


V 

< 

c 

§ 

CO 

o 
O 


en 

a; 

•^ 

c 

ca 

O     0) 

£ 

o 

e  § 

'■4-» 

-a   c 

OQ 

aj  ^ 

U3 

c    ca 

■e 

■^    «S 

CO      4) 

0)  -C 

e 

c;  t3 

0) 

S 

-o   a, 

^s 

^  E 

O   J= 

6 

notjusti 
displace 

2J  o 

Ql       CO 

3  >:  E 

gests 
otype 

chniq 

mates 
;e  the 

c  -^ 

M  "S 

0>    .=    ^ 

3    2 

cfi   a. 

•J  w    ca 

<  ^  £ 

I2X) 

■4.J 

.*J 

■*j 

t-    c 

« •> . 

•^ 

.1.J 

J£ 

c 

c 

G 

c: 

Ided  as 
d  by  mar 

<u 

V 

«     «   "en 

v 

0) 

T3 
C 

a 

c 

T3 

d  valu 
n  achi 
sgoal 

c 

ID 
01 

c 

Si 

■o 

0> 

a 

4> 

OJ  ■-    en 

^   «        1 

> 

>    en    a> 

-^ 

rg 

CO    u          1 

■—    en    c 

(J    5    en 

£ 

E 

Value 
measu 

en 

u 

u    3    3 

*-t-j 

"■^ 

CL.    en  : 

7! 

en 

.2 

'b. 

CO 

u 

> 

C 

-a 

c 

a 

0) 

-a 
c 

en 

o 
o 

-o 

ca 

£ 

-4-> 

en 

O 
o 

-o 

■•-> 

ca 
E 

3 
O 

JO 

ca 

0) 

u 

> 
o 

o 

•^ 

■*-) 

.t^ 

en 

en 

**J 

'*j 

en 

o 

O 

Cfi 

en 

o 

O 

O 

C3L] 

U 

O 

■u 

u 

4-> 

o 

a; 
en 

CO 

1 

"c 
o 

u 

bene 
seat 

en    a) 
>>  C 

u 

-^     3 

anal 
St/be 

*j 

o  -n 

0} 

3  o 

i^ 

r"?^     3 

1 
1 

c   c 

u     U 

(U    o 

CO    o 

"^     en 

pilot  a 
on  of  a 
tDEC 

logy  fo 
ixampl 

ts  using  valu 
of  rigorous c 

si 

■Z3    ca 
ca    en 

uating 
d,  mea 

udy  of 
lentati 
stem  a 

o     ^     ■Ji 

-C     ea     ^ 

tevalu 
tionsa 

tseval 
e  adde( 
place 

«  1  ^ 

01  _  _ 

1  -55  O 

en  "a 

0)    ca 

en     3 

4)     (J 

en     3    -7^ 

&£    01 

ac::: 

aj)  ca  -^ 

en    D.  •- 

6*  -S 

M  a. 

M  >    b 

CO    c    S 
0.5    E 

2    §J 

3     ^ 

3    a. 
c/2   ca 

c^   §   E 

C^I 

OI 

"O 

eXl 

00 

c    - 

CJ) 

< 

1 

3 

million  ai 
urn,  1985 

c" 

•n 

71 

ea 

01 

0)   j2 

ca 

o 

:s 

u: 

C 

iy5 

CO 


CO 

ca 

C 


£ 

o 

c 
o 


Comments 

3 

"a 

1 
1 

Used  frontier  analysis  to 
calculate  efficiency 

Used  DEA  to  calculate 
efficiency  and  rates  of 
substitution 

Used  DEA  to  calculate 
efficiency  of  each  department 

c 
41 
> 

'So 

03 

_4) 

£ 

ca 

X 
4> 

0 
Z 

tn 

4) 

.2 

C8 

> 

c 

C 
4> 

Return  on  assets  and  net 
worth,  profits,  net 
growth 

Efficiency  in 
transforming  production 
capital  and  labour  to 
labour  costs  +  profits 

Return  on  investment 
and  total  sales 

User  information 
satisfaction 

c 

4) 

> 

4) 

c 
0 

Z 

Level  of  economic 
growth 

c 
4; 

-0 
c 

4) 

a 

4> 
•a 

c 

Ownership  of 
computers,  number  of 
applications  used 

Number  of  non- 
production  workers, 
expenditures  on  IT, 
number  of  applications 
used 

Capital,  labour  and  IT 
investment 

IT  investment, 
employee  satisfaction, 
performance,  task 
complexity  and 
technology  level 

01 
>. 

_2 
a> 

Q 

c 

c 

4J 

£ 
01 

4) 

> 

c 

03 

.2 
'S 

S. 

E 

8 

V 

1 

1 

00 

0 

> 
u 
3 
1/3 

03 

.2 
'c 
« 

a. 

E 
0 

u 

c 

2 
'So 

i 

u 

0 
2 

00 

e« 
O 
>> 

41 
> 
u 

70 

Description  of  DEA  with  numeric 

example 

en 

c 
£ 

L. 
!9 
0. 
9i 

-a 

a. 
/-\ 

0 

w- 
0 
>. 

> 
u 
3 

cn 

Suggests  using  system  dynamic 
simulations  and  gaming 

Use  microeconomics  to  estimate 
impact  of  information  technology 
on  efilciency  and  a  macro-economic 
model  to  estimate  impact  on 
economy 

— 

<: 

Cron  and  Sobol, 
1983 

00 
!73 

J2 

Chismar  and 
Kriebe  1.1985 

Klam. 

Henderson  and 
Thomas,  1984 

en 
r- 

i 

CO 
00 
03 

u 
u 
j= 
u 

03 

c 
0 

n 

OS 


remaining  articles  are  framework  or  theory 
development  pieces. 

Survey  of  Studies  of  Enterprise 
Level  Performance 

There  are  a  number  of  articles  that  attempt  to 
develop  techniques  for  identifying  strategic 
systems  opportunities.  As  is  characteristic  in  a 
new  field  of  study,  many  of  these  articles  propose 
frameworks.  They  try  to  establish  a  clear  vision  of 
possible  alternative  systems  by  employing  a  cate- 
gorization scheme  built  using  important  dime- 
nsions of  the  problem.  DiiTerent  ones  have  been 
proposed  by  Bakopoulos  and  Treacy  (1985), 
Barrett  and  Konsynski  (1982),  Heath  and  Ives 
(1986),  Benjamin  e<a/ (1983),  Cash  and  Konsynski 
(1985),  Gerstein  and  Reisman  (1982),  Ives  and 
Learmouth  (1984),  Keen  (1981b),  McFarlan 
(1984),  Parsons  (1983),  and  Rockart  and  Scott 
Morton  (1984).  Through  descriptive  work  we  have 
developed  a  fair  degree  of  understanding  of  what 
the  rEinge  of  systems  possibilities  and  impacts  are. 
It  is  now  time  for  academic  research  to  contribute 
explanations  of  how  these  systems  impact  com- 
petition and  corporate  performance.  If  we  are  to 
correctly  influence  managerial  practice,  we  need 
to  understand  how  internal  support  systems 
contribute  to  enterprise  level  performance,  and 
how  that  performance  should  be  defined  and 
measured. 

COST/BENEFIT  ANALYSES 

Some  authors  have  begun  to  address  these  issues. 
One  group  of  articles  generally  suggests  doing 
some  form  of  cost/benefit  analysis  to  "justify"  an 
organization's  investment  in  information  tech- 
nology. These  articles,  based  as  they  are  on  a 
simple  accounting  view  of  productivity,  are  per- 
haps the  most  common  and  the  least  informative. 
(It  should  be  noted  that  in  some  of  these  articles,  a 
costA)enefit  analysis  is  presented  as  a  small  part  of 
a  larger  paper.) 

Crawford  (1982)  describes  the  pilot  test  and  actual 
implementation  of  an  electronic  mail  system  at 
DEC,  and  presents  some  figures  on  the  costs  and 
benefits  obtained.  These  figures  show  that  the 
system  can  not  be  justified  solely  on  the  grounds  of 
cost  displacement,  but  that  improved  managerial 
performance  must  be  considered  as  well 

Others,  recognizing  the  increasing  difficulty  of 
assigning  a  precise  value  to  the  intangible  benefits 
of  information  systems,  suggest  new  ways  to  es- 


timate or  defend  estimates  of  benefits.  Matlin 
(1979)  advocates  assigning  values  to  IT  projects 
based  on  how  they  achieve  "business  goals"  and 
describes  such  an  evaluation  done  at  Land 
O'Lakes.  Keen  (1981a)  suggests  using  a  technique 
he  calls  "value  analysis"  to  justify  decision  support 
systems  rather  than  trying  to  rigorously  calculate 
their  costs  and  benefits.  He  suggests  first  deve- 
loping a  prototype  system,  which  can  be  con- 
sidered R&D  and  thus  not  rigorously  justified. 
After  the  potential  benefits  of  the  system  are 
clearer,  an  assessment  of  the  final  implementation 
can  be  made,  and  a  rigorous  cost/benefit  analysis 
done  only  if  the  estimated  benefit  is  not  obviously 
greater  than  the  cost.  Gremillion  and  Pyburn 
(1985)  suggest  evaluating  a  portfolio  of  appli- 
cations as  a  whole  rather  than  trying  to  defend  the 
estimated  benefits  of  each  individual  system. 
Strassman  (1982)  suggest  calculating  a  system's 
"effectiveness"  by  dividing  the  value  it  adds  (the 
market  value  of  the  final  product  less  the  input 
costs)  by  the  overhead  labour  cost. 

Most  of  these  articles  are  think-pieces,  presenting 
a  methodology  for  cost/benefit  analysis,  but  little 
or  no  real  data.  Of  those  that  do  give  empirical 
results,  two  (Crawford  and  Matlin)  are  case 
studies  of  the  organization  for  which  the  author 
worked.  While  these  studies  are  interesting,  they 
have  limited  external  validity;  knowing  that  DEC 
or  Land  O'Lakes  believe  that  they  are  gaining 
benefit  from  their  systems  tells  us  very  little  about 
any  other  company.  The  results  are  generalizable 
only  if  they  are  based  on  characteristics  shared  by 
many  other  firms.  We  have  no  idea  what  features 
of  these  firms  allow  their  systems  to  be  successful 
while  many  other  systems  fail.  This  is  a  problem 
of  internal  and  external  validity,  and  it  is 
compounded  by  weak  operationalizations  of  IT. 
These  studies  generally  measure  the  presence  or 
absence  of  IT  but  do  not  assess  the  level  and  type  of ' 
use  to  which  it  is  put.  Therefore,  it  is  difficult  to 
assess  the  mechanism  by  which  the  system  afTects 
enterprise  performance. 

Yet  another  problem  with  cost/benefit  based 
studies  is  their  weak  concept  of  performance. 
These  studies  try  to  identify  benefit,  but 
unfortunately,  it  is  unclear  what  the  benefits  due 
to  a  system  are  and  there  are  no  accepted  methods 
for  assessing  them.  The  actual  benefit  obtained 
may  be  different  from  that  expected,  change  over 
time  or  even  diiTer  for  difi'erent  users  (Ginzberg, 
1979).  The  best  these  articles  can  suggest  is  that 
this  is  a  difficult  problem  that  must  be  faced 
(Matlin),  avoided  (Keen)  or  left  to  the  market 
(Strassman).  Given  this  combination  of  idio- 
syncratic and  poorly  operationalized  measures  and 


case  analyses,  it  is  not  surprising  that  there  have 
been  few  solid  results  to  date.  There  will  be  little 
progress  in  this  area  without  a  better  choice  of 
variables. 

One  attempt  in  this  direction  was  made  by 
Ginzberg  (1979),  who  developed  a  list  of  nine  types 
of  benefits,  based  on  a  study  of  project  proposals 
and  justifications.  Unfortunately,  many  of  his  cat- 
egories are  rather  vague  (e.g.,  promote  organiza- 
tional learning)  and  difHcult  to  operationalize 
without  a  better  theoretical  base,  a  point  to  which 
we  will  return  later. 


ECONOMIC  ANALYSES 

A  second  group  of  authors  try  to  measure 
performance  by  applying  methodologies  and  defi- 
nitions of  performance  drawn  from  economics. 
Cron  and  Sobol  (1983)  attempt  to  relate  the 
performance  of  wholesale  companies,  measured  by 
return  on  assets,  return  on  net  worth,  profits  as  a 
percentage  of  sales,  and  average  growth,  to  their 
use  of  IT,  measured  by  ownership  of  computers 
and  number  of  software  capabilities  used.  Their 
results  are  inconclusive,  showing  that  heavy  users 
of  IT  tend  to  be  either  higher  or  lower  performers 
than  average. 

Many  authors  have  suggested  measures  of 
technical  efficiency  drawn  from  microeconomics. 
These  authors  compute  how  well  each  firm  does 
with  its  resources  by  using  some  form  of  efficient 
frontier  analysis.  Stabell  and  Forsund  (1983) 
relate  a  firm's  use  of  computers  to  its  efficiency. 
They  first  use  frontier  analysis  to  calculate  the 
efficiency  of  82  large  Norwegian  companies  who 
used  computers,  taking  as  input  the  number  of 
production  workers  and  capital,  and  as  output  the 
sum  of  total  labour  costs  and  net  profits.  They 
then  relate  the  estimated  efficiency  to  measures  of 
information  systems  use,  such  as  the  number  of 
non-production  employees  (both  total  and  as  a 
ratio  of  total  employees,  a  ratio  Stabell  calls 
"administrative  intensity"),  expenditures  on  EDP 
(total  and  as  a  percentage  of  sales),  number  of 
different  applications,  and  number  of  terminals 
(total  and  per  100  non-production  employees). 
Their  results  seems  to  show  that  efficiency  is 
unrelated  to  firm  size  or  any  absolute  measure  of 
use  of  IT,  but  is  correlated  with  the  "admin- 
istrative intensity"  and  to  the  relative  DP 
expenditures. 

Chismar  and  Kriebel  (1985)  suggest  using  a  type 
of  frontier  analysis  called  data  envelopment 
analysis  (DEA)  and  time-series  data  to  measure 


efficiency.  They  model  the  firm's  inputs  as 
investment  in  information  technology,  non- 
production  labour  and  capital  and  production 
labour  and  capital  and  provide  a  numerical 
example  of  the  technique,  using  return  on 
investment  and  total  sales  as  output.  They  dem- 
onstrate how  the  technique  can  be  used  to  estimate 
technical  rates  of  substitution  between  input 
factors,  such  as  I/S  investment  and  non-production 
labour.  They  also  mention  some  unresolved 
problems  with  DEA,  such  as  the  difficulty  in 
choosing  what  to  measure  as  inputs  and  outputs, 
and  the  scarcity  of  usable  data.  The  first  concern 
is  especially  troubling  since  the  inputs  and  out- 
puts are  used  as  a  characterization  of  the  activity 
and  performance  of  the  firm. 

Elam,  Henderson  and  Thomas  (1984)  view  the 
information  systems  group  itself  as  a  production 
unit,  taking  inputs  and  producing  some  output. 
They  use  DEA  to  assess  how  successful  10  data 
centres  are  in  providing  user  information  sat- 
isfaction given  inputs  of  money,  technology  level, 
IS  employee  satisfaction  and  performance,  and 
task  complexity. 

Another  approach  to  the  problem  of  productivity  is 
the  creation  of  an  explicit  model  of  the  system. 
This  methodology  has  the  advantage  that  any 
assumptions  about  the  system  must  be  made 
explicitly  and  the  effect  of  changing  them  can  be 
quickly  determined.  Kleijnen  (1979)  advocates 
using  system  dynamics  models,  and  suggests 
using  simulations  and  laboratory  games  to 
investigate  effects  on  productivity.  At  a  much 
higher  level  of  analysis,  Jonscher  (1983)  uses  a 
macroeconomic  model  of  the  U.S.  economy  to 
predict  that  the  effect  of  the  estimated  level  of 
investment  in  information  technology  will  reverse 
the  slowdown  of  economic  growth  by  the  1980's. 
(We  will  have  to  wait  to  see  if  this  prediction  has 
been  fulfilled.)  Jonscher's  conclusions  rest  on  an 
estimate  of  the  effect  of  information  inputs  on 
economic  output  through  improvements  in  the 
efficiency  of  production  and  trading  functions. 
This  estimate  is  based  on  microeconomic  argu- 
ments, which  were  calibrated  for  use  at  the 
national  economy  level. 

Many  of  these  studies  done  at  the  firm  level  using 
economic  methodologies  seem  to  suffer  from  much 
the  same  data  problems  as  the  cost/benefit  anal- 
yses. To  calculate  efficiency,  these  studies  need  to 
measure  the  firm's  inputs  and  outputs,  but  there  is 
little  agreement  about  what  these  should  be  or 
how  to  measure  them.  Most  of  these  studies  use 
very  blunt  and  aggregate  measures,  taking  as 
inputs  such  variables  as  the  total  investment  in 


information  systems  or  the  amount  of  non- 
production  labour.  As  an  output,  Elam,  Hender- 
son and  Thomas  use  user  information  satisfaction, 
but  it  is  unclear  that  this  relates  to  firm  pro- 
ductivity. Studies  at  the  individual  level  obtain 
correlations  between  job  satisfaction  and  per- 
formance of  less  than  .2  (Vroom,  1964).  Other 
studies  use  financial  performance  indicators  such 
as  return  on  assets  or  total  sales.  These  variables 
are  very  aggregate  products  of  the  firms' 
accounting  system,  and  are  not  closely  related  to 
information  technology  impacts.  To  show  a  meas- 
urable change  in  these  variables,  an  information 
system  would  have  to  have  a  huge  impact  at  some 
lower  level.  Once  again,  progress  in  this  area  will 
be  slow  until  we  have  a  better  idea  about  which 
performance  variables  to  choose  and  how  to 
measure  them. 


the  case  with  systems  applied  to  routine  problems 
to  reduce  costs  (e.g.,  by  reducing  clerical  labour). 
For  the  less  routine  applications  being  studied 
today,  an  accounting  view  can  give  no  advice 
about  which  variables  to  consider  as  inputs  or 
outputs,  explaining  the  prevalence  of  idiosyncratic 
measures  in  this  area. 

Furthermore,  since  the  process  is  ignored,  there  is 
no  way  to  logically  link  the  chosen  inputs  to  the 
outputs.  Even  a  perfect  cost/benefit  analysis  could 
only  tell  you  the  benefit  derived  from  the  system. 
It  could  not  suggest  if  the  benefits  of  one  system 
were  more  worthwhile  than  those  of  other 
systems,  if  you  are  doing  better  or  worse  than 
others,  or,  most  importantly,  what  you  could  or 
should  be  doing  instead.  This  final  problem  was 
recognized  by  Matlin,  who  wrote: 


Reference  Disciplines 

The  following  admittedly  simplistic  model  of  the 
firm  (see  Figure  1)  is  a  useful  way  to  organize  our 
discussion  of  the  underlying  reference  disciplines. 
The  model  illustrates  simply  that  firms  take  some 
inputs,  perform  some  processes,  and  produce  some 
outputs.  The  articles  we  have  discussed  in  the 
first  half  of  this  paper  investigate  link  1  in  this 
figure.  The  difference  between  the  reference 
disciplines  used  by  these  authors  affects  how  the 
research  is  approached  and  what  is  put  in  the 
process  box. 


Many  companies  follow  the  similar  pattern  in  their 
search...  companies  apply  controls  to  the  largest, 
obvious  expense  activity-the  computer.  Cost  and 
efficiency  measures  are  reflected  in  the  request  for  and 
production  of  data  relating  to  equipment  utilization, 
productivity,  and  responsiveness....  these  control 
measures  are  satisfied  quickly.  Even  though  these 
controls  are  established  and  reporting  satisfactory 
performance,  management  is  not  satisfied....  After 
spending  a  considerable  amount  of  time  and  energy  in 
these  control  activities,  senior  managers  find  that  they 
are  still  frustrated  in  their  attempts  to  to  feel  good 
about  their  management  of  the  information  systems 
activity....  (p.  33) 


MICROECONOMICS 


ACCOUNTING 

The  first  set  of  articles  are  based  on  some  notion  of 
productivity  drawn  from  accounting.  These 
studies  basically  ignore  the  process  box  altogether. 
Instead,  they  attempt  to  sum  up  the  additional 
inputs  (the  cost)  and  the  outputs  (the  benefits)  and 
check  that  the  output  (the  benefit)  is  greater  than 
the  input  (the  cost),  or  that  the  system  added  some 
value.  This  approach  is  only  satisfactory  when  the 
benefits  are  large  and  obvious  to  compute,  as  was 


The  second  set  of  articles  we  discussed  draws  on 
methodologies  from  microeconomics.  Here  the 
process  that  links  inputs  to  outputs  is  modelled, 
but  very  simply  using  computed  ratios  of  input  to 
output  transformation.  Since  we  know  nothing 
specific  about  the  process,  we  find  ourselves  in  the 
same  position  as  before  when  we  come  to  choose 
variables.  Literally  anything  could  be  an  input  or 
an  output  of  the  process,  and  we  have  no  theory  to 
guide  our  choice.  Furthermore,  treating  the 
process  as  a  "black  box"  implies  a  static  view  of  the 
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firm.  It  is  difficult  for  such  an  economic  approach 
to  model  changes  in  the  processes  in  the 
organization,  possibly  confounding  any  such  study 
(Cohen,  1984).  Kleijnen  and  Jonscher  both 
suggest  using  some  type  of  simulation  model  to  fill 
the  process  box,  but  even  so  their  work  is  not  tied 
to  any  particular  theory  of  the  firm,  leaving  their 
choice  of  variables  still  somewhat  ad  hoc. 

Filling  the  process  box  with  a  theory  that  links 
inputs  to  outputs  (i.e.,  that  investigates  the  links 
labeled  with  a  2  in  Figure  1)  has  some  obvious 
advantages  that  recommend  it  as  an  approach  for 
future  research.  First,  the  process  theory  should 
clarify  which  inputs  and  outputs  of  the  firm  are 
important  and  may  even  contribute  methodologies 
to  measure  them.  Second,  by  explicitly  including 
the  processes  within  the  firm,  we  can  look  at  the 
impacts  of  IT  in  much  more  detail.  Instead  of 
standing  outside  and  attempting  to  pick  out  small 
variations  in,  for  example,  return  on  investment, 
we  can  look  at  where  IT  directly  impacts  the  firm 
and  make  a  much  more  precise  estimate  of  this 
impact.  Finally,  and  most  importantly,  we  can 
discover  the  contingencies  that  allow  systems  to 
affect  firm  performance,  and  prescribe  the  features 
of  systems  that  will  be  useful  to  particular  firms. 
As  Ginzberg  said: 

It  is  only  once  we  understand  how  the  new  information 
will  be  used  that  its  value  can  be  estimated.  Thus, 
efforts  to  quantify  benefits  should  focus  on  the  changes 
in  organizational  process  which  will  result  from 
changes  to  information  systems,  (p.  535) 

Many  different  theories  about  organizations  could 
be  used  to  fill  the  process  box.  One  obvious  source 
for  such  theories  is  the  field  of  strategy.  In  a  rough 
sense,  strategic  performance  is  concerned  with 
long-term  profits,  which  can  be  achieved  either 
through  superior  revenues  or  superior  cost 
performance.  The  utility  of  partitioning  strategic 
performance  into  these  two  components  is  that  a 
body  of  literature  within  industrial  economics  and 
corporate  strategy  relates  to  each,  namely 
monopolization  theory  and  Williamson's  theory  of 
transaction  cost  economics  (1975,  1983).  These 
two  fields  are  obvious  places  to  look  for 
foundational  theory  for  studying  the  impact  of 
information  systems  on  enterprise  performance 
and  they  provide  methodologies  which  could  serve 
us  well  in  these  studies. 


MARKET  POWER 

The  industrial  economic  theory  of  market  power  or 
monopolization  theory,  provides  a  basis  for  under- 
standing the  effects  of  information  technology  on 


prices,  market  share,  and  revenues.  Monopoly 
power  is  enhanced  through  attractive  product  dif- 
ferentiation and  by  reducing  the  amount  of 
searching  for  suppliers  performed  by  customers. 
Information  technology  can  affect  both  these 
variables.  For  example,  product  differentiation 
can  be  achieved  by  bundling  IT  with  existing 
products  to  differentiate  them  from  competition. 
The  size  of  a  customer's  search  for  suppliers  can  be 
affected  with  direct  order  entry  systems  and  other 
forms  of  vertical  information  integration.  This 
strategy  has  been  employed  most  notably  by 
American  Hospital  Supply  (Harvard,  1985;  Petre, 
1985). 

A  supplier's  monopoly  power  can  be  reduced  by 
avoiding  unique,  differentiated  products  and  by 
searching  widely  for  competing  suppliers.  The 
economics  of  searching,  which  directly  affect  the 
size  of  search  set,  are  often  radically  altered  with 
information  technology.  In  fact,  electronic  market: 
places,  much  like  a  stock  exchange,  can  reduce  the 
cost  of  searching  for  the  most  economical  supplier 
nearly  to  zero.  This  facilitates  finding  the  best 
product  at  the  best  price  and  reduces  any  price 
premium  that  the  supplier  might  otherwise  have 
extracted  from  the  firm.  For  example,  electronic 
reservation  systems  such  as  American  Airlines 
SABER  system  have  reduced  the  differentiation 
between  airlines  and  made  it  possible  for 
customers  to  quickly  select  the  best  flight, 
regardless  of  airline.  As  a  result,  sales  by  travel 
agencies  have  jumped  from  35%  to  70%  of  the 
total,  and  American  now  makes  more  money 
running  SABER  than  they  do  running  an  airline 
(Petre,  1985). 

TRANSACTION  COST  ECONOMICS 

Williamson's  studies  of  markets  and  hierarchies 
can  help  to  explain  the  enterprise  and  industry- 
level  impact  of  information  technology  by  explain- 
ing changes  in  production  and  transaction  costs. 
He  points  out  that  the  boundaries  between  indus- 
tries arise  at  those  points  where  a  market's  advan- 
tage of  production  efficiencies  outweigh  the 
transaction  cost  superiority  of  internal  organiza- 
tion. Simply  put,  separate  and  specialized  indus- 
tries exist  because  at  some  points  it  is  cheaper  to 
buy  a  product  or  service  than  to  make  it. 
Williamson's  model  has  been  used  to  study  the 
degree  of  vertical  integration  in  the  automobile 
manufacturing  industry  (Monteverde  and  Teece, 
1982;  Walker  and  Weber,  1984)  and  the  decision  to 
forward  integrate  with  a  direct  sales  force  versus 
using  manufacturers'  representatives  (Anderson. 
1982). 


Information  technology  has  the  potential  to 
radically  alter  cost  structures  and  transform  the 
structure  of  industry  boundaries.  In  some  cases, 
functions  that  were  once  integrated  into  the  firm 
may  be  eliminated  and  alternatives  may  be 
purchased  in  a  market.  In  other  cases,  products 
and  services  that  were  once  purchased  now  may  be 
created  by  functions  within  the  firm.  IT  can  have 
this  impact  on  industry  structure  by  altering  the 
relative  production  efficiencies  and  transaction 
costs  of  market  and  organization  mechanisms,  and 
the  specificity  of  assets  that  create  products. 

INFORMATION  PROCESSING 

Another  useful  source  of  theories  for  future 
investigations  of  enterprise  level  performance 
impacts  is  the  information  processing  view  of 
organizations  (Galbraith,  1974;  Cohen,  1984). 
This  view  includes  many  ideas  drawn  from 
Williamson's  analysis,  but  goes  beyond  it  by 
attempting  to  uncover  the  content  of  transactions 
and  the  requirements  of  their  processing.  A  few 
articles  have  been  published  using  this  view  to 
link  productivity  to  internal  features  of  the  firm. 
These  authors  view  the  firm  as  an  information 
processor,  with  a  resulting  focus  on  organizational 
units  as  processors  that  communicate  information 
between  themselves  within  the  firm.  Huber 
(1982)  summarizes  existing  research  related  to 
information  handling  and  identifies  four  key 
processes  operating  in  organizational  information 
systems-message  routing,  summarizing,  delay 
and  modification-and  presents  a  number  of  pro- 
positions about  each.  Malone  and  Smith  (1984) 
show  how  the  structure  of  a  firm  affects  the  ways 
in  which  information  can  be  exchanged  between 
its  subunits.  Using  queueing  theory,  they  calcu- 
late the  relative  efficiency,  flexibility  and  vulner- 
ability of  several  simple  organizational  forms. 
Malone  (1985)  extends  this  analysis  to  incorporate 
elements  of  other  organizational  theories  and 
shows  how  an  information  processing  view  can  be 
used,  for  example,  to  explain  historical  changes  in 
organizational  structure.  Benjamin,  Malone,  and 
Yates  (1986)  discuss  how  IT  may  increase  the  use 
of  markets  for  coordination,  rather  than  decisions 
within  a  firm. 

The  information  processing  view  has  a  number  of 
features  that  make  it  useful  as  a  process  theory  for 
studying  enterprise  level  performance  impacts 
First,  information  processing  is  ideally  suited  for 
interpreting  the  effects  of  IT  on  organizations 
because  it  explicitly  addresses  the  ability  of 
computers  and  humans  to  process  information 
This  ability  is  in  many  ways  orthogonal  to 
features  central  to  other  organizational  theories. 


For  example,  it  is  unclear  what  effect  an  electronic 
mail  system  will  have  on  the  power  structure  of 
groups  in  an  organization,  but  more  clear  what  it 
will  do  to  their  ability  to  communicate.  The  effect 
of  IT  is  better  reflected  through  certain  reference 
theories  than  through  others. 

Second,  although  more  encompassing,  the 
information  processing  view  still  has  ties  to 
traditional  microeconomics  and  to  transaction 
economics,  suggesting  the  possibility  of  borrowing 
some  definitions  and  methodologies  from  these 
more  developed  fields.  The  definition  of  technical 
efficiency  used  in  some  of  the  studies  reviewed 
above  could  be  employed,  for  example,  to  examine 
input  and  output  variables  suggested  by  the  infor- 
mation processing  view.  Block  modelling  may  be  a 
useful  method  for  determining  communications 
patterns  within  a  group. 

Finally,  information  processing  suggests  and 
permits  the  use  of  organizational  simulations,  as 
is  also  suggested  by  Kleijnen.  Simulations  have  a 
number  of  advantages  for  research  that  are  de- 
sirable in  this  area.  First,  simulations  require 
that  assumptions  be  made  explicit,  making  them 
easier  to  see  and  the  results  of  changing  them 
easier  to  test.  Second,  simulations  make  it  pos- 
sible to  analyze  systems  that  are  too  complex  for 
analytic  solution. 


CONCLUSION 

The  three  theories  we  have  discussed,  market 
power  theory,  transaction  cost  economics  and  the 
information  processing  view  of  organizations, 
identify  variables  through  which  we  can  study  the 
impact  of  information  technology  on  enterprise 
performance.  Market  power  theory  suggests  out- 
put variables  related  to  consumers'  search  for  pro- 
ducts, such  as  the  number  of  similar  products 
available,  the  number  actually  considered,  or  the 
method  used  to  search  for  new  products. 
Transaction  cost  economics  highlights  the  cost  of 
transactions  between  entities  such  as  customers, 
firms  or  divisions.  Williamson  identifies  features 
of  the  environment  that  affect  these  transactions 
and  shows  how  they  interact  with  differences  in 
production  costs  to  change  the  relative  advantages 
of  market  and  internal  production.  The  infor- 
mation processing  view  includes  many  of  these 
transaction  variables,  but  looks  more  closely  at 
the  content  and  processing  of  transactions  within 
as  well  as  between  firms.  This  suggests  examin- 
ing more  closely  the  pattern  of  communication  (or 
who  talks  to  whom),  and  the  specific  processing 
that  needs  to  be  done  and  the  messages  required. 


Once  we  have  chosen  a  specific  process  theory,  we 
can  identify  interesting  input  and  output 
variables  and  begin  to  investigate  more  precisely 
the  possible  impacts  of  IT.  For  this  investigation, 
we  can  use  methodologies  proven  in  other  fields, 
such  as  technical  efficiency  and  DEA  from  micro- 
economics, or  different  ones  suggested  by  the  new 
reference  discipline,  such  as  simulations  or  games. 


Suggested  Research  Directions 


So  far  we  seem  to  have  learned  little  about  the 
impacts  of  IT  on  enterprise  level  productivity. 
There  have  been  a  number  of  studies  based  on 
accounting  data,  either  performing  a  cost/benefit 
analysis  or  using  methodologies  borrowed  from 
microeconomics.  These  studies,  however,  are 
flawed  by  a  lack  of  theory  about  the  processes 
within  the  firm.  This  absence  leaves  the  selection 
of  measures  of  performance  somewhat  arbitrary. 
Furthermore,  although  operational  systems  have 
been  the  focus  of  most  of  the  work  done  in  MIS  in 
the  past,  IT  is  now  viewed  as  a  strategic  tool, 
dictating  a  longer-term  view  of  the  impacts  of  IT. 
This  change  in  focus  makes  it  even  more 
important  to  develop  sensitive  measures  of 
performance,  based  on  strong  reference  disciplines 
and  useful  theories  about  the  processes  within 
firms.  Research  must  be  able  to  make 
prescriptions  about  the  features  of  systems  and 
organizations  that  contribute  to  successful  uses  of 
IT,  as  well  as  recognize  them  after  they  have 
happened.  Unfortunately,  most  of  what  has  been 
written  about  the  strategic  use  of  IT  has  lacked 
this  strong  theoretical  foundation.  We  have  seen, 


however,  a  few  promising  reference  disciplines, 
such  as  industrial  economics  and  the  information 
processing  view  that  could  guide  future  research.     ' 

The  need  for  a  strong  reference  discipline  becomes 
even  clearer  when  we  look  at  the  literature  with 
the  framework  suggested  earlier  (see  Figure  2).  It 
is  clear  that  there  are  still  a  large  number  of 
fundamental  yet  unanswered  questions.  Most  of 
these  questions  remain  unanswered  because  there 
is  no  way  to  address  them  based  solely  on 
empirical  studies  of  the  inputs  and  outputs,  the 
form  of  the  majority  of  the  studies  to  date.  Instead, 
we  must  look  for  a  strong  theory  about  the 
processes  in  organizations  to  guide  our  choice  of 
variables  and  to  generate  testable  hypothesis 
about  them.  Without  such  a  theory,  we  will  be 
faced  with  far  too  many  possible  input  or  output 
variables  and  no  way  to  control  for  the  many 
interactions  between  them.  For  example  in  the 
cases  of  AHS  and  SABER  mentioned  above,  it  is 
clear  how  the  use  of  IT  is  affecting  the  process  of 
searching  for  suppliers  and  the  effect  this  change 
has  on  the  performance  of  the  company. 

Once  we  have  chosen  a  reference  discipline  and 
thus  our  variables  of  interest,  we  can  borrow 
accepted  definitions  and  well  tested  methodologies 
to  do  more  systematic  and  valid  studies.  These 
studies  should  cover  more  firms  but  at  a  more 
specific  level:  rather  than  trying  to  use  blunt 
measures  like  existence  of  a  computer  system  or 
return  on  investment,  we  can  look  at  precise 
measures  of  the  inputs  and  outputs.  This  suggests 
starting  with  a  better  typology  of  organizational 
processes  and  the  possible  impacts  of  IT  and 
working  from  there  towards  productivity. 
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